The invention relates to hardware implementations of one-dimensional filters applied to digital images such as are supplied from digital cameras, especially applying multiple such filters including recursive filters to a stream of digital images. More specifically, the resulting filtered digital image may then be processed in software on a general-purpose computer to complete a two-dimensional digital image filter.
Digital image processing filters are common in the electronics and computer industries. Applications include medical imaging, video conferencing, computer graphics, and satellite surveillance; in all these areas it is necessary to process digital images quickly and cheaply. Image processing ASIC""s are often highly suitable for such applications. Another useful function is image compression, which reduces the data content of a given image at a cost of a possible loss of resolution or an introduction of artifacts. This data reduction is a critical step in sending images over a data channel of limited bandwidth.
One type of image processing is the generation of an image pyramid, which represents an image at multiple scales. This form of image representation is useful for isolating features of a specific scale and to implement coarse-to-fine algorithms. In addition, it may be possible to reduce image data bandwidth by transmitting a coarser level of an image pyramid if the original, full-resolution image is not required. Specialized hardware for generating full pyramids are described in U.S. Pat. No. 5,359,674 to van der Wal. That system generates pyramids in hardware using a separable filter, consisting of two separate passes of a 1-D filter. These pyramid chips require large line delay memories integrated into the ASIC itself to provide the necessary intermediate storage for the final 2-D filter results. Consequently, the filtering cannot be done using standard single chip ASIC""s, but instead require specialized hardware. Therefore, the hardware costs for such a system are relatively expensive.
There is a need for a digital image filtering system that can perform 2-D filtering by simply modifying a typical existing single chip ASIC core, such as a digital camera controller ASIC. Such a system should be able to produce the same results as the pyramid processor without requiring either large ASIC""s with substantial on-chip memories or multiple chip implementations. This system would be substantially less expensive than the pyramid processor.
We provide a reliable method and apparatus for integrating one or more one-dimensional digital image filters into a single image stream. In our method we only perform one step of the two-pass full pyramid generation process in hardware by producing one-dimensional partial-pyramided results. Then we perform the second pass in software. The input to the first pass consists of the full-resolution original images, or only the odd or even raster lines of a video image, and are performed in hardware for maximum speed. The input to the second pass consists of partially-pyramided images, which are smaller than the original full-resolution images, and therefore can be efficiently transmitted via a (typically high-bandwidth) digital transmission interface to a host computer""s memory and processed in software at video rates by an inexpensive general-purpose CPU. These smaller, partially-pyramided images which are input to the second pass require correspondingly fewer computational cycles than the larger images input to the first pass, making a software implementation of the second pass feasible.
As a result of this combination of hardware and software approaches we do not require large line-delay memories as an intermediate storage for the final full 2-D filter results which would be necessary for a hardware-only approach such as is disclosed in the van der Wal patent. This results in a substantial savings in hardware (VLSI chip real-estate) and thus cost. The hardware requirements are designed to be sufficiently low, that this one-dimensional filtering can be added onto existing digital interface ASIC""s without requiring multiple-chip implementations. Even though much less hardware is used than in devices performing two-dimensional filtering, the majority of the computational cycles required for this process can be performed one-dimensionally; resulting in a much higher performance-cost ratio. In addition, our device is able to interleave multiple one-dimensional filter results, including recursive one-dimensional filter results, into a single output stream (whose components can be separated by an appropriate decoder) without requiring on-chip memory to store the intermediate filter results. Thus, the only on-chip video memory required, other than that required for the one-dimensional digital image filtering itself, is any buffering necessitated by the high-bandwidth transmission interface to compensate for latency inherent to the transmission medium.
The image output can be connected to a high-bandwidth transmission interface such that each image filter result can be automatically extracted from the image stream. The extracted images can be placed into a separate buffer in a host computer""s image memory in a form suitable for subsequent additional filtering, especially the second pass of a two-pass separable digital image filter. This combination of hardware and software filtering represents a significant cost and size savings over a hardware-only implementation while still providing high performance.